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ABSTRACT 

Some  practical  sufficient  conditions  are  given  for  a  program  with 
linear  constraints  and  nonlinear  objective  to  have  well-behaved  duality 
properties.  Thus  if  the  objective  is  convex,  lower  semicontinuous , 
and  satisfies  a  Lipschitz  condition  on  a  closed  convex  polyhedron,  and 
is  +“  elsewhere,  then  there  exists  a  dual  problem  which  actually 
achieves  the  optimal  value  of  the  primal  problem.  An  argument  is  given 
suggesting  that  certain  of  the  results  are  best  possible  from  the 
viewpoint  of  practical  applications.  These  results  are  applied  to 
show  that  a  broad  class  of  stochastic  programs  with  recourse  have 
desirable  duality  properties. 


1.  INTRODUCTION 


Theorem  1  below  gives  sufficient  conditions  for  a  program  with  linear 
constraints  and  a  convex  objective  with  range  in  the  extended  reals  to 
have  well-behaved  duality  properties.  The  principal  ingredient  in  the 
proof  of  Theorem  1  is  Theorem  2,  which  considers  the  properties  of  the 
optimum  value  of  a  (not  necessarily  convex)  program  under  perturbation 
of  the  linear  constraints.  The  terminology  used  in  Theorem  1  is  adapted 
from  [1,2,3]  where  the  close  relationship  between  properties  of  the 
variational  function  and  the  duality  properties  of  a  program  have  been 
discussed  at  some  length.  Briefly  a  program  is  solvable  if  the  value 
of  the  infimum  is  finite  and  achieved  for  some  value  of  the  variables, 
it  is  dualizable  if  there  is  no  duality  gap,  and  it  is  stable  if  there 
exist  (optimal)  nontrivial  Lagrange  multipliers.  The  definitions  of 
convexity  and  lower  semicontinuity  for  functions  into  the  extended 
reals  will  be  reviewed  in  the  next  section. 

At  the  end  of  §2  an  argument  will  be  given  suggesting  that  Theorem 
1  is  the  best  possible  from  the  viewpoint  of  practical  applications. 
Finally,  Theorem  1  will  be  applied  in  §4  to  show  that  a  broad  class 
of  stochastic  programs  with  recourse  have  desirable  duality  properties. 

THEOREM  1.  Consider  the  nonlinear  program 
Inf  f(x) 


Ax  ■  b 


(1) 


where  the  objective  f  is  convex  in  the  sense  of  functions  into  the 
extended  reals. 

(i)  If  f  is  lower  semi  continuous  In  the  sense  of  functions  into 
the  extended  reals  and  the  constraint  set  K.  ■  {x|Ax  •  b,  x  1  0} 
is  bounded  and  contains  at  least  one  point  where  f  is  finite , 
then  (1)  is  solvable  and  dualizable. 

(ii)  If  f  is  finite  and  Lipschitz  on  a  closed  convex  polyhedron  and 
(1)  has  a  finite  value,  then  (1)  is  stable. 

THEOREM  2.  Consider  the  function  (u)  ■  Inf{f(x)|x  e  k(u)}, 


where 

tc(u)  - 

lx 

|Ax  ■  b  -  u,  x  1  0}  and 

f  is  a  function  with  range 

in  the 

extended 

reals . 

(i) 

If  f 

is 

convex,  so  is  $. 

(ii) 

If  f 

is 

lcwer  semi  continuous  and 

k(u)  is  compact  for  some 

u,  then  <p  is  lower  semi  continuous . 

(iii)  If  f  is  +“  except  on  some  closed  convex  polyhedron  where  it 
is  either  finite  and  Lipschitz  or  identically  then  the 

same  holds  for  <J>. 

2.  PROOF  OF  THEOREM  2 

DEFINITIONS.  Let  f  be  a  function  with  Rn  for  domain  and  the 
extended  reals  R  ■  R  U  {+“}  U  {“”}  for  range.  The  set 

epi  f  -  {(z,x)  |  z  e  R,  x  e  Rn,  z  £  f(x)} 

is  called  the  epigraph  of  f.  The  function  f  is  said  to  be  convex  if 
its  epigraph  is  a  convex  subset  of  Rn+1  or  equivalently  if 


f(x^)  -  f [(I-aJxq+AXj]  <  (1-X) f (xQ)  +  XfCx^ 
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for  all  A  e  [0,1]  ami  xQ,Xj  c  Rn,  where  the  conventions  0  •  00  ■  0 

and  (+®)  +  (-®)  apply.  The  function  f  is  said  to  be  lower 

8emicontinuous  if 

lim  inf  f(x^)  1  f(lim  x^) 

for  every  convergent  sequence  (x^)  in  Rn,  or  equivalently  if  epi  f 
is  a  closed  subset  of  Rn+1.  If  the  function  f  is  finite  on  some 
subset  S  of  Rn  and  B  is  a  real  number  such  that 

(f(x)  -  f (x* ) |  <  B  | |x  -  x* | | 

for  all  x,x'  in  X,  where  ||*||  is  the  Euclidean  norm  in  Rn,  then 
f  Is  said  to  be  Llpschitz  on  S  with  constant  B. 

The  following  two  lemmas  represent  an  application  of  results  in 
[6]  to  die  special  situation  of  Theorem  2.  (An  inconsequential  difference 
is  that  in  [6]  k(u)  is  a  section  of  P  rather  than  its  projection  into 
R°.) 

LEMMA  1.  Let  P  ■  {  (x,u) | Ax  +  Du  ■  b ,  x  >  0} ,  k  (u)  • 

{x  |  Ax  -  b  -  Du,  x  >  0}  ,  Q  -  {u  |  k(u)  is  nonempty}  ,  and  let  P  ■  PQ  +  C 

be  the  representation  of  the  polyhedron  P  as  the  vector  sum  of  a  bounded 
polyhedron  PQ  and  a  (unique)  polyhedral  cone  C  with  apex  at  the 
origin.  Then  Q  is  a  polyhedron,  and  for  each  u  e  Q,  tc(u)  ■  <Q(U)  +  C', 

where  kq(u)  is  a  bounded  polyhedron  depending  on  u  and  C'  is  the 

polyhedral  con>i  { x |  ( x ,0)  e  C }.  In  particular,  k(u)  is  bounded  for 

all  u  in  Q  if  it  is  bounded  for  some  u  in  Q,  for  in  this  case 
C'  -  {o}. 
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LEMMA  2.  Let  <  and  Q  be  defined  as  in  Lemma  1.  Then  there 
exists  a  constant  B  such  that  for  any  two  points  and  in  Q 

d[x(uj) ,k(u2) ]  i  B  | |uj-u2  |  | , 

where  d[  ,  ]  denotes  the  Hausdorff  distance  between  sets  in  Rn. 

Observe  that  Lemmas  1  and  2  apply  directly  to  Theorem  2  if  u  is 
taken  to  be  a  vector  in  Rm  and  the  matrix  D  is  the  Identity. 
Accordingly  let  P  and  Q  be  defined  in  this  manner,  and  consider 
part  (i)  of  Theorem  2.  New  P  is  convex,  and  by  assumption  epi  f  is 
a  convex  subset  of  R  x  Rn.  Hence  (epi  f  x  Rm)  O  R  x  p  is  a  convex 

subset  of  R  x  Rn  x  Rm  and  its  projection  onto  the  space  R  x  Rm  is 
a  convex  set  e  •  But  epi  $  is  just  the  vertical  closure  of  6, 

i.e.,  the  union  of  C  and  any  missing  endpoints  of  "vertical"  line 

segments  in  C .  This  proves  part  (i)  of  Theorem  2.  Alao,  Q  is 

closed  and  4>(u)  ■  +®  if  u  is  not  in  Q.  Thus,  in  order  to  prove 

part  (ii)  of  Theorem  2  it  suffices  to  show 

lim  inf  0(u1)  >  <p(u0) , 

where  u^  e  Q  and  lim  u^  -  uQ  e  Q.  By  Lemma  1  each  <(u)  is  compact, 
and  hence  the  lower  semicontinuous  function  f  attains  a  minimum  on 
k(u)  at  some  point,  say  x(u) .  Also,  it  follows  from  Lemma  2  that 
k(u)  is  uniformly  bounded  for  u  in  some  compact  neighborhood  N  of 
uQ,  i.e.,  P  O  [R"  lxN]  is  compact.  Hence  the  sequence  of  points  x(u^) 
has  at  least  one  limit  point  xQ  t  k(uq).  Then 


4 
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lim  inf  $(0^  -  lim  inf  f(x(u1>)  >  f(xQ)  1  f(x(uQ))  -  $(uQ)  , 

where  the  first  inequality  follows  from  the  lower  semicontinuity  of 
f  and  the  second  follows  from  the  optimality  of  x(uQ)  on  *c ( uQ) . 

This  proves  part  (ii)  of  Theorem  2. 

We  shall  begin  the  proof  of  part  (iii)  of  Theorem  2  by  establishing 
the  following  special  case: 

LEMMA  3.  Under  the  conditions  of  Theorem  2,  if  f  is  finite 
and  Lipschltz  throughout  Rn,  then  either  $  is  identically 
on  Q  or  if  is  finite  and  Lipschltz  on  Q. 

Proof.  Let  u  and  u'  be  any  two  points  of  Q,  let  (x^)  be 
a  sequence  of  points  in  k(u)  such  that  lim  l'(x^)  -  <f>(u),  and  let 

x|  be  the  point  of  k(u')  closest  to  x^.  Then 

*(u’)  -  f(Xl)  <  f(xj)  -  f(x±) 

^  B  ||xj  -  xi||  (2) 

<  BB  ||u*  -  u| | 

where  B  is  the  Lipschltz  constant  for  f  and  B  is  the  constant  of 
Lemma  2.  Now  4>(u')  may  be  -«  or  finite,  and  lim  f(x^)  -  q>(u>  may 
be  -<*>  or  finite.  But  (2)  shews  that  $(u)  ■  -«  implies  <f>(u')  ■ 
i.e.,  4>  is  identically  -®  on  Q  or  $  is  finite  on  Q.  And  if 

4>  is  finite  on  Q,  then  (2)  implies  $(u')  -  4>(u)  <  BB  ||u'  —  u |  |  .  By  the 

symmetry  in  u  and  u'  it  follows  that  $  is  Lipschltz  with  constant  b¥. 

Now  suppose,  as  assumed  in  Theorem  2,  that  the  range  of  f  is 


h 


I 


\ 
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contained  in  Che  extended  reals,  and  the  set  K  -  { x |  f ( x)  <  +»} 
is  a  closed  convex  polyhedron.  Then  $  may  be  defined  by  the 
program 

#(u)  ■  Inf  f(x) 

Ax  ■  b  -  u 

(3) 

A'x  >  0 
x  e  K 

Since  K  is  a  polyhedron,  (3)  may  be  rewritten 

#(u)  “  Inf  f(x)  +  Ox' 

Ax  -  b  -  u  (A) 

A'x  +  x'  -  b' 
x  >  0,  x'  >0 

The  set  K0  -  |u|$(u)  <  +00}  is  exactly  the  set  of  u  for  which 
the  constraints  of  (3)  or  (4)  are  feasible,  hence  this  set  is  a  poly¬ 
hedron.  Moreover,  if  f  is  ■»  on  K,  then  <f>  is  on  KQ. 

Thf s  establishes  a  portion  of  part  (iii)  of  Theorem  2.  The  remaining 
possibility  is  that  f  is  finite  and  Llpschltz  on  K.  Now  f  may  not 
be  finite  and  Lipschitz  where  (4)  is  infeasible,  and  u  does  not  perturb 
.  all  constraints  of  (4),  but  clearly  the  proof  of  Lemma  3  will  apply  to 
(4),  with  a  different  choice  of  D  in  Lemma  1,  and  yield  the  remainder 
of  Theorem  2. 

Strictly  speaking  the  requirement  that  |x|f(x)  <  +®}  be  a  polyhedron 


is  not  essential  to  Theorem  2.  The  proof  of  Lemma  3  clearly  establishes 
the  following  more  general  result. 
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LEMMA  4.  Suppose  f  is  finite  and  Lipschitz  everywhere,  K 
is  a  closed  convex  subset  of  Rn  *  Rm,  <f>(u)  ■  Inf  { f (x)  |x  e  k(u)}, 
k(u)  -  {x|  (x,u)  e  K},  and  Q  ■  {u|k(u)  is  nonempty}.  Then  $  is 
either  -®  on  Q  or  finite  and  Lipschitz  on  Q  provided  there  exists 
a  constant  B  such  that 

d[<(u),K(u')J  i  B  | [ u  -  u* | |  (5) 

for  all  u,u'  in  Q. 

However,  it  is  difficult  to  see  how  this  more  general  result  can 
be  applied  to  practical  problems.  For  consider  a  program  of  the  form 

4>(u)  ■  Inf  f(x) 

Ax  ■  b  -  u 
x  e  K 

where  K  is  a  convex  set.  Surely  any  practical  condition  to  be 

imposed  on  P  ought  not  to  depend  on  the  particular  form  of  A.  But 

it  is  shown  in  [6]  that  if  P  is  not  a  polyhedron  there  even  exist 

ai,...,a  such  that 
1  n 

k(uj)  ■  {x  |  ajXj+. .  .+anxn  ■  -Uj ,  x  e  P} 
fails  to  satisfy  (5)  for  any  B. 

3.  DERIVATION  OF  THEOREM  1 

For  easy  reference,  we  repeat  the  convex  program  of  Theorem  1: 
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Inf  f(x) 

Ax  *  b  (1) 

x  l  0 


The  basic  duality  properties  of  this  program  are  conveniently  represented 
in  an  equivalent  Inf  problem 


Inf  n 

(o  ,u)  e  £  n  e 


(6) 


where  £  is  the  vertical  line  {(n,u)|n  e  R,  u  ■  0}  and  6  is  the 
convex  set  whose  vertical  closure  is  the  epigraph  of  <p,  as  described 
in  the  proof  of  part  (i)  of  Theorem  1.  The  program  (1)  will  be  solvable , 

i.e.,  have  a  finite  optimum  value  <K0)  "  f(x°)  achieved  by  some  feasible 

0 

x  ,  if  and  only  if  £  D  C  is  a  closed  half  line. 


A  natural  dual  to  the  Inf  problem  (6)  is  the  Sup  problem: 


Sup 

(u,u*) 


u 

u  1  n  +  u*u  for  all  (n,u)  c  C. 


The  Inf  problem  asks  for  the  infimal  height  n  of  points  on  £ 
inside  C.  The  Sup  problem  asks  for  the  supremal  height  u  of  points 
on  £  through  which  can  be  passed  nonvertical  hyperplanes  (i.e.,  those  not 
containing  any  line  parallel  to  £)  bounding  Q.  The 
Sup  problem  can  be  recast  in  the  form 


Sup  U 

(  U,u*) 

0  <  f(x)  +  u*(b-Ax)  for  all  x  ±  0 


where  the  role  of  u*  as  a  row  m-vector  of  Lagrange  multipliers  is 
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appa. eat . 

It  is  natural  to  say  that  the  Inf  and  Sup  problems  are  dual  or 
that  (1)  is  dualizable  if  Sup  p  ■  Inf  n»  i.e.,  if  there  is  no  duality 
gap.  We  shall  say  that  the  program  (1)  is  stable  if  the  Sup  problem  is 
solvable,  i.e.,  has  a  finite  optimum  which  is  achieved  for  some  u*. 

Thus  (1)  is  stable  if  there  is  a  nonvertical  hyperplane  supporting  G 
at  the  point  (<p(0)  ,0)  on  £.  Equivalently,  (1)  is  stable  if  there 
exist  Lagrange  multipliers  which  convert  it  into  an  equivalent  uncon¬ 
strained  problem. 

The  above  discussion  of  duality  properties  of  (1)  follows  the  approach 
to  mathematical  programming  in  abstract  spaces  given  in  [3]  which  is 
closely  related  to  the  more  extensive  development  given  by  Rockafellar 
in  [1,2].  Theorem  1  now  follows  easily  from  the  above  discussion, 
well-known  properties  of  convex  sets  and  their  supports,  the  simple 
relationship  between  G  and  the  epigraph  of  $,  and  from  Theorem  2. 

REMARK.  We  have  seen  that  (1)  is  stable  if  f  satisfies  a 
Lipschltz  condition.  But  also,  the  duality  theory  for  quadratic 
programs  shows  that  (1)  is  stable  if  f  is  convex  and  quadratic,  and 
hence  definitely  not  Lipschitz.  We  do  not  know  of  any  sensible  condition 
on  f  which  implies  both  properties  and  is  valid  in  Theorem  1.  A  simple 
example  will  show  that  (1)  may  fail  to  be  stable  if  K  is  unbounded  and 
f  is  only  locally  Lipschitz  on  a  polyhedron. 
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4.  APPLICATIONS  TO  STOCHASTIC  PROGRAMS  WITH  RECOURSE 

ir.  ciiis  section  we  will  use  ilieorem  1  to  show  that  fairly  broad 
classes  of  stochastic  programs  with  recourse  have  well-behaved  duals. 

The  necessary  results  on  the  objective  functions  of  such  programs  have 
already  been  proved  in  earlier  papers  [4,5].  We  shall  make  liberal  use 
of  the  notation  and  terminology  introduced  in  [4,5,8]. 

THEOREM  3.  If  the  value  of  a  stochastic  program  with  recourse 
is  finite  and  the  first-stage  feasibility  set  Kj  *  {x|ax  =  b,  x  -  o} 
is  bounded,  then  the  equivalent  convex  program  is  solvable.  Moreover, 
either  the  equivalent  convex  program  is  dualizable  or  the  objective  takes 
the  vaiue  -■  at  points  belonging  to  K1  for  arbitrarily  smull  pertur¬ 
bation!  of  b,  i.e.,  there  is  an  Infinite  duality  gap. 

Proof .  It  is  shown  in  [5]  that  the  objective  z(x)  of  the 

equivalent  convex  program  is  either  lower  semicontinuous  as  a  function 

into  the  extended  reals  or  takes  the  value  -«  at  some  point.  The 

second  part  of  the  theorem  follows  immediately  from  this  and  Theorem  1. 
s 

Now  let  K2  be  the  set  on  which  z(x)  is  less  than  +°°  and  let  M 

g 

be  the  affine  hull  of  the  intersection  of  Kj  and  K2 .  Since  z  is 
s  s 

convex,  so  is  K., ,  and  hence  K  H  K  has  an  interior  with  respect  to 

*-  1  2 

M.  Since  the  restriction  z^  of  z  to  M  is  convex,  and  since 

it  is  finite  on  KjH  K® ,  it  follows  that  it  is  nowhere  -«.  The 

results  of  [5]  apply  equally  well  to  z..  and  show  that  it  is  lowe- 

M 

semicontinuous.  A  straightforward  application  of  Theorem  1  completes 
the  proof. 
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THEOREM  4.  If  the  value  of  a  stochastic  program  with  recourse 
is  finite,  the  recourse  matrix  W  is  fixed,  the  second-stage  feasibility 
set  K2  is  a  polyhedron,  and  the  random  variables  £  have  finite 
variance,  then  the  equivalent  convex  program  is  stable. 

Proof.  Theorem  (4.5)  of  [4]  shows  that  under  the  second  and 
fourth  hypotheses  either  the  objective  of  the  equivalent  convex  program 
is  -°°  throughout  K2  or  it  is  finite  and  Lipschitz  on  K2.  The  rest 
follows  from  Theorem  1. 

It  is  worth  mentioning  that  Proposition  (3.16)  of  [4]  gives  practical 
conditions  which  insure  that  K2  is  a  polyhedron.  In  addition  Corollary 
(4.7)  of  [4]  gives  some  alternate  conditions  on  the  distribution  of  £ 
under  which  the  conclusions  of  Theorem  (4.5),  and  hence  the  above  theorem, 
remain  valid. 

We  conclude  with  a  few  remarks  showing  that  the  hypotheses  of 
Theorems  3  ar  d  4  are  not  superfluous.  Consider  the  special  class  of 
stochastic  programs  considered  in  detail  in  [7],  for  which  W  ■  [1,-1] 
and  only  the  right-hand  sides  are  random.  It  is  easy  to  construct 
examples  in  this  class  such  that  the  random  variables  have  a  distribution 
possessing  all  moments,  and  the  value  of  the  infimum  is  finite,  but  the 
program  is  not  solvable,  provided  Kj  is  allowed  to  be  unbounded.  Thus 
boundedness  of  Kj  is  not  superfluous  for  solvability  in  Theorem  3.  We 
remark  that  in  s.ich  an  example  z(x)  will  be  finite  everywhere,  and 
hence  by  Theorem  4  such  an  example  will  be  stable,  but  not  solvable. 
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A  stochastic  program  can  be  constructed  with  T  only  random  and 
K,  bounded  for  which  the  alternative  at  the  end  of  Theorem  3  occurs. 

An  example  can  also  be  constructed  u . .h  T  only  random  and  Kj 
unbounded  for  which  z(x)  >  -®,  and  hence  z(x)  is  lower  semi continuous , 
but  f(u)  still  jumps  from  to  a  finite  value  at  u  *  0.  Thus 

boundedness  of  is  not  superfluous  for  dualizability  in  Theorem  3. 

We  do  not  know  whether  a  stochastic  program  with  recourse  can  exhibit 
a  finite  duality  gap;  this  is  certainly  possible  for  a  program  of  type 
(1)  in  which  f(x)  is  convex  and  lower  semicontinuous  but  K  is 
unbounded  [3,  p.  692]. 

It  can  be  shown  by  example  that,  as  the  remarks  at  the  end  of  §2 
should  suggest,  the  requirement  that  be  a  polyhedron  is  not  super¬ 

fluous  in  Theorem  4.  Finally  a  variant  of  the  example  found  in  [5]  will 
show  that  finite  variance  of  t  is  not  superfluous  in  Theorem  4. 
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